Variational joint self‐attention for image captioning

The image captioning task has attracted great attention from many researchers, and significant progress been made in the past few years. Existing models, which mainly apply attention-based encoder-decoder architecture, achieve developments captioning. These however, are limited caption generation due to potential errors resulting inaccurate detection of objects incorrect objects. To alleviate limitation, a Variational Joint Self-Attention model (VJSA) is proposed learn latent semantic alignment between given its label description for guiding better Unlike existing VJSA first uses self-attention module encode effective relationship information intra-sequence inter-sequences relationships. And then variational neural inference learns distribution over corresponding description. In decoding, learned guides decoder generate higher quality caption. results experiments reveal that outperforms compared performances various metrics show feasible generation.